Use identifier in HarvesterGroup in filestream#24864

Merged

kvch merged 1 commit intoelastic:masterfrom

kvch:fix-filebeat-sourcestore-in-harvestergroup

Mar 31, 2021

Contributor

kvch commented Mar 31, 2021 •

edited

Loading

What does this PR do?

This PR adds sourceIdentifier to defaultHarvesterGroup so it uses the same method to access entries in the registry as the Prospector.

Why is it important?

Previously, the HarvesterGroup and Prospector looked for the same resources but got different IDs. HarvesterGroup was not aware of the filestream::{user_id} prefix, thus the Prospector never found the entries created by the HarvesterGroup and vica versa.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~- [ ] I have made corresponding changes to the documentation~~
~~- [ ] I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
~~- [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.~~

botelastic Bot added the needs_team label

kvch requested a review from urso

March 31, 2021 09:21

kvch added the Team:Elastic-Agent label

Contributor

elasticmachine commented Mar 31, 2021

Pinging @elastic/agent (Team:Agent)

botelastic Bot removed the needs_team label

Contributor

elasticmachine commented Mar 31, 2021 •

edited by jenkins-beats-ci Bot

Loading

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #24864 updated
Start Time: 2021-03-31T11:37:37.660+0000
Duration: 54 min 32 sec
Commit: 356f306

Test stats 🧪

Test	Results
Failed	66
Passed	13127
Skipped	2271
Total	15464

Trends 🧪

Test errors

Expand to view the tests failures

> Show only the first 10 test failures

`Build&Test / filebeat-build / test_fileset_file_060_traefik – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_060_traefik>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_060_traefik>
evt = {'@timestamp': '2017-10-02T20:22:07.000Z', 'agent': {'ephemeral_id': 'f6c89931-6565-4fec-9698-5bc5f5969283', 'id': '4d...category': ['web'], 'created': '2021-03-31T12:10:03.116Z', 'dataset': 'traefik.access', 'duration': 2000000, ...}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_061_nginx – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_061_nginx>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_061_nginx>
evt = {'@timestamp': '2020-02-07T11:48:51.000Z', 'agent': {'ephemeral_id': 'b2a9df23-091e-4edd-8dd1-4ef95759aea3', 'id': '4d...1-03-31T12:10:05.012Z', 'dataset': 'nginx.ingress_controller', 'ingested': '2021-03-31T12:10:06.190916609Z', ...}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_062_nginx – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_062_nginx>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_062_nginx>
evt = {'@timestamp': '2016-12-07T10:05:07.000Z', 'agent': {'ephemeral_id': 'a6fb3dd4-aaa7-4b36-9a81-d836c5e0b262', 'id': '4d...eated': '2021-03-31T12:10:07.072Z', 'dataset': 'nginx.access', 'ingested': '2021-03-31T12:10:08.186200071Z', ...}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_063_nginx – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_063_nginx>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_063_nginx>
evt = {'@timestamp': '2016-10-25T12:49:33.000Z', 'agent': {'ephemeral_id': 'f017c9f6-740c-40c1-bbb3-fe13d2b8f37c', 'id': '4d...eated': '2021-03-31T12:10:08.890Z', 'dataset': 'nginx.access', 'ingested': '2021-03-31T12:10:09.953999630Z', ...}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_064_nginx – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_064_nginx>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_064_nginx>
evt = {'@timestamp': '2016-12-07T10:05:07.000Z', 'agent': {'ephemeral_id': '79726272-7870-4f1e-99c3-368189299daa', 'id': '4d...: '2a1232683a1e', 'type': 'filebeat', ...}, 'destination': {'domain': 'example.com'}, 'ecs': {'version': '1.8.0'}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_067_iis – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_067_iis>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_067_iis>
evt = {'@timestamp': '2018-01-01T08:09:10.000Z', 'agent': {'ephemeral_id': 'dc707599-d3a4-4164-924b-1835bbca09f4', 'id': '4d...ebeat', ...}, 'destination': {'address': '127.0.0.1', 'ip': '127.0.0.1', 'port': 80}, 'ecs': {'version': '1.8.0'}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_068_iis – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_068_iis>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_068_iis>
evt = {'@timestamp': '2018-08-28T18:24:25.000Z', 'agent': {'ephemeral_id': '7fa89695-4d7d-4854-8018-6b4fff29ae99', 'id': '4d......}, 'destination': {'address': '10.100.220.70', 'ip': '10.100.220.70', 'port': 80}, 'ecs': {'version': '1.8.0'}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_069_iis – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_069_iis>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_069_iis>
evt = {'@timestamp': '2020-10-04T22:00:34.000Z', 'agent': {'ephemeral_id': '8d180a3e-22c5-4ce6-84d6-85f5e129f7f8', 'id': '4d....162', 'domain': 'images.hogeschoolrotterdam.nl', 'ip': '10.24.129.162', 'port': 80}, 'ecs': {'version': '1.8.0'}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_070_iis – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_070_iis>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_070_iis>
evt = {'@timestamp': '2018-12-31T12:02:53.000Z', 'agent': {'ephemeral_id': 'db124396-144f-4531-8da9-e2dd6999fa13', 'id': '4d..., ...}, 'destination': {'address': '10.44.0.136', 'ip': '10.44.0.136', 'port': 8080}, 'ecs': {'version': '1.8.0'}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

`Build&Test / filebeat-build / test_fileset_file_071_iis – filebeat.tests.system.test_modules.Test`

Expand to view the error details

 Exception: Key 'user_agent.device.type' found in event is not documented!

Expand to view the stacktrace

 a = (<test_modules.Test testMethod=test_fileset_file_071_iis>,)

    @wraps(func)
    def standalone_func(*a):
>       return func(*(a + p.args), **p.kwargs)

../build/ve/docker/lib/python3.7/site-packages/parameterized/parameterized.py:518: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/system/test_modules.py:99: in test_fileset_file
    cfgfile=cfgfile)
tests/system/test_modules.py:183: in run_on_file
    self.assert_fields_are_documented(obj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_modules.Test testMethod=test_fileset_file_071_iis>
evt = {'@timestamp': '2020-10-07T23:00:17.000Z', 'agent': {'ephemeral_id': '53a36009-cf39-4adf-a7a7-2ea20e3ce7eb', 'id': '4d.....}, 'destination': {'address': '192.168.16.11', 'ip': '192.168.16.11', 'port': 443}, 'ecs': {'version': '1.8.0'}, ...}

    def assert_fields_are_documented(self, evt):
        """
        Assert that all keys present in evt are documented in fields.yml.
        This reads from the global fields.yml, means `make collect` has to be run before the check.
        """
        expected_fields, dict_fields, aliases = self.load_fields()
        flat = self.flatten_object(evt, dict_fields)
    
        def field_pattern_match(pattern, key):
            pattern_fields = pattern.split(".")
            key_fields = key.split(".")
            if len(pattern_fields) != len(key_fields):
                return False
            for i in range(len(pattern_fields)):
                if pattern_fields[i] == "*":
                    continue
                if pattern_fields[i] != key_fields[i]:
                    return False
            return True
    
        def is_documented(key, docs):
            if key in docs:
                return True
            for pattern in (f for f in docs if "*" in f):
                if field_pattern_match(pattern, key):
                    return True
            return False
    
        for key in flat.keys():
            metaKey = key.startswith('@metadata.')
            # Range keys as used in 'date_range' etc will not have docs of course
            isRangeKey = key.split('.')[-1] in ['gte', 'gt', 'lte', 'lt']
            if not(is_documented(key, expected_fields) or metaKey or isRangeKey):
>               raise Exception("Key '{}' found in event is not documented!".format(key))
E               Exception: Key 'user_agent.device.type' found in event is not documented!

../libbeat/tests/system/beat/beat.py:729: Exception

Steps errors

Expand to view the steps failures

`filebeat-build - mage build test`

Took 21 min 18 sec . View more details on here
Description: mage build test

`x-pack/filebeat-build - mage build test`

Took 33 min 24 sec . View more details on here
Description: mage build test

`Error signal`

Took 0 min 0 sec . View more details on here
Description: Error 'hudson.AbortException: script returned exit code 1'

Log output

Expand to view the last 100 lines of log output

[2021-03-31T12:28:39.089Z]   Go version:       go1.13.15
[2021-03-31T12:28:39.089Z]   Git commit:       46229ca
[2021-03-31T12:28:39.089Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-03-31T12:28:39.089Z]   OS/Arch:          linux/amd64
[2021-03-31T12:28:39.089Z]   Experimental:     false
[2021-03-31T12:28:39.089Z]  containerd:
[2021-03-31T12:28:39.089Z]   Version:          1.4.4
[2021-03-31T12:28:39.089Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-03-31T12:28:39.089Z]  runc:
[2021-03-31T12:28:39.089Z]   Version:          1.0.0-rc93
[2021-03-31T12:28:39.089Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-03-31T12:28:39.089Z]  docker-init:
[2021-03-31T12:28:39.089Z]   Version:          0.19.0
[2021-03-31T12:28:39.089Z]   GitCommit:        de40ad0
[2021-03-31T12:28:39.089Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-03-31T12:28:39.089Z] Unable to find image 'alpine:3.4' locally
[2021-03-31T12:28:40.036Z] 3.4: Pulling from library/alpine
[2021-03-31T12:28:40.036Z] c1e54eec4b57: Pulling fs layer
[2021-03-31T12:28:40.299Z] c1e54eec4b57: Download complete
[2021-03-31T12:28:40.563Z] c1e54eec4b57: Pull complete
[2021-03-31T12:28:40.563Z] Digest: sha256:b733d4a32c4da6a00a84df2ca32791bb03df95400243648d8c539e7b4cce329c
[2021-03-31T12:28:40.563Z] Status: Downloaded newer image for alpine:3.4
[2021-03-31T12:28:43.117Z] Change permissions with write access of all files inside the specific folder
[2021-03-31T12:28:44.086Z] Running in /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8/src/github.com/elastic/beats/build
[2021-03-31T12:28:44.393Z] + rm -rf ve
[2021-03-31T12:28:44.393Z] + find . -type d -name vendor -exec rm -r {} ;
[2021-03-31T12:28:44.729Z] + python .ci/scripts/pre_archive_test.py
[2021-03-31T12:28:47.287Z] Copy ./x-pack/filebeat/build into build/x-pack/filebeat/build
[2021-03-31T12:28:47.302Z] Running in /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8/src/github.com/elastic/beats/build
[2021-03-31T12:28:47.320Z] Recording test results
[2021-03-31T12:28:49.149Z] [Checks API] No suitable checks publisher found.
[2021-03-31T12:28:49.478Z] + tar --version
[2021-03-31T12:28:49.811Z] + tar --exclude=test-build-artifacts-x-pack-filebeat-build-tgz -czf test-build-artifacts-x-pack-filebeat-build-tgz .
[2021-03-31T12:29:46.604Z] + gsutil --version
[2021-03-31T12:29:46.604Z] /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8/src/github.com/elastic/beats/build@tmp/durable-3e86d842/script.sh: 1: /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8/src/github.com/elastic/beats/build@tmp/durable-3e86d842/script.sh: gsutil: not found
[2021-03-31T12:29:46.705Z] Running in /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8/src/github.com/elastic/beats/build@tmp
[2021-03-31T12:29:47.055Z] + wget --version
[2021-03-31T12:29:47.436Z] + wget -q -O gsutil.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-linux-x86_64.tar.gz
[2021-03-31T12:29:48.438Z] + tar -xpf gsutil.tar.gz --strip-components=1
[2021-03-31T12:29:52.231Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-03-31T12:29:52.726Z] + gcloud auth activate-service-account --key-file ****
[2021-03-31T12:29:54.672Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-03-31T12:29:54.999Z] + gsutil -m -q cp -a public-read test-build-artifacts-x-pack-filebeat-build-tgz gs://beats-ci-temp/Beats/beats/PR-24864-3
[2021-03-31T12:30:00.671Z] + python .ci/scripts/search_system_tests.py
[2021-03-31T12:30:00.692Z] [INFO] system-tests='build/x-pack/filebeat/build/system-tests'. If no empty then let's create a tarball
[2021-03-31T12:30:01.042Z] + tar --version
[2021-03-31T12:30:01.364Z] + tar --exclude=x-pack-filebeat--system-tests-linux-tgz -czf x-pack-filebeat--system-tests-linux-tgz build/x-pack/filebeat/build/system-tests
[2021-03-31T12:30:33.849Z] + gsutil --version
[2021-03-31T12:30:33.849Z] /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8/src/github.com/elastic/beats@tmp/durable-6bbd6ace/script.sh: 1: /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8/src/github.com/elastic/beats@tmp/durable-6bbd6ace/script.sh: gsutil: not found
[2021-03-31T12:30:33.902Z] Running in /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8/src/github.com/elastic/beats@tmp
[2021-03-31T12:30:34.229Z] + wget --version
[2021-03-31T12:30:34.558Z] + wget -q -O gsutil.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-linux-x86_64.tar.gz
[2021-03-31T12:30:35.144Z] + tar -xpf gsutil.tar.gz --strip-components=1
[2021-03-31T12:30:37.762Z] Masking supported pattern matches of $FILE_CREDENTIAL
[2021-03-31T12:30:38.083Z] + gcloud auth activate-service-account --key-file ****
[2021-03-31T12:30:39.500Z] Activated service account credentials for: [beats-ci-gcs-plugin@elastic-ci-prod.iam.gserviceaccount.com]
[2021-03-31T12:30:40.098Z] + gsutil -m -q cp -a public-read x-pack-filebeat--system-tests-linux-tgz gs://beats-ci-temp/Beats/beats/PR-24864-3
[2021-03-31T12:30:44.705Z] + go clean -modcache
[2021-03-31T12:30:48.446Z] Cleaning up /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8
[2021-03-31T12:30:48.447Z] Client: Docker Engine - Community
[2021-03-31T12:30:48.447Z]  Version:           20.10.3
[2021-03-31T12:30:48.447Z]  API version:       1.41
[2021-03-31T12:30:48.447Z]  Go version:        go1.13.15
[2021-03-31T12:30:48.447Z]  Git commit:        48d30b5
[2021-03-31T12:30:48.447Z]  Built:             Fri Jan 29 14:33:13 2021
[2021-03-31T12:30:48.447Z]  OS/Arch:           linux/amd64
[2021-03-31T12:30:48.447Z]  Context:           default
[2021-03-31T12:30:48.447Z]  Experimental:      true
[2021-03-31T12:30:48.447Z] 
[2021-03-31T12:30:48.447Z] Server: Docker Engine - Community
[2021-03-31T12:30:48.447Z]  Engine:
[2021-03-31T12:30:48.447Z]   Version:          20.10.3
[2021-03-31T12:30:48.447Z]   API version:      1.41 (minimum version 1.12)
[2021-03-31T12:30:48.447Z]   Go version:       go1.13.15
[2021-03-31T12:30:48.447Z]   Git commit:       46229ca
[2021-03-31T12:30:48.447Z]   Built:            Fri Jan 29 14:31:25 2021
[2021-03-31T12:30:48.447Z]   OS/Arch:          linux/amd64
[2021-03-31T12:30:48.447Z]   Experimental:     false
[2021-03-31T12:30:48.447Z]  containerd:
[2021-03-31T12:30:48.447Z]   Version:          1.4.4
[2021-03-31T12:30:48.447Z]   GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
[2021-03-31T12:30:48.447Z]  runc:
[2021-03-31T12:30:48.447Z]   Version:          1.0.0-rc93
[2021-03-31T12:30:48.447Z]   GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
[2021-03-31T12:30:48.447Z]  docker-init:
[2021-03-31T12:30:48.447Z]   Version:          0.19.0
[2021-03-31T12:30:48.447Z]   GitCommit:        de40ad0
[2021-03-31T12:30:48.447Z] Change ownership of all files inside the specific folder from root/root to current user/group
[2021-03-31T12:30:52.659Z] Change permissions with write access of all files inside the specific folder
[2021-03-31T12:30:53.263Z] Running in /var/lib/jenkins/workspace/PR-24864-3-624ab469-2e53-4726-8124-ccaf036c9de8
[2021-03-31T12:30:59.388Z] Failed in branch x-pack/filebeat-build
[2021-03-31T12:30:59.497Z] Stage "Packaging" skipped due to earlier failure(s)
[2021-03-31T12:30:59.581Z] Running in /var/lib/jenkins/workspace/Beats_beats_PR-24864/src/github.com/elastic/beats
[2021-03-31T12:31:09.131Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats_PR-24864
[2021-03-31T12:31:09.183Z] [INFO] getVaultSecret: Getting secrets
[2021-03-31T12:31:09.283Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-03-31T12:31:10.037Z] + chmod 755 generate-build-data.sh
[2021-03-31T12:31:10.037Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24864/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24864/runs/3 FAILURE 3212110
[2021-03-31T12:31:10.287Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24864/runs/3/steps/?limit=10000 -o steps-info.json
[2021-03-31T12:31:11.631Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats/PR-24864/runs/3/tests/?status=FAILED -o tests-errors.json

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	66
Passed	13127
Skipped	2271
Total	15464

Genuine test errors

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

Name: Build&Test / filebeat-build / test_fileset_file_060_traefik – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_061_nginx – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_062_nginx – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_063_nginx – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_064_nginx – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_067_iis – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_068_iis – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_069_iis – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_070_iis – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_071_iis – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_072_iis – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_106_apache – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_107_apache – filebeat.tests.system.test_modules.Test
Name: Build&Test / filebeat-build / test_fileset_file_108_apache – filebeat.tests.system.test_modules.Test
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_048_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_049_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_051_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_052_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_053_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_054_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_055_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_056_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_057_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_058_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_059_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_060_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_061_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_062_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_063_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_064_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_065_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_066_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_067_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_068_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_069_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_070_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_071_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_072_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_073_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_074_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_075_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_076_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_077_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_078_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_079_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_081_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_082_aws – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_131_zeek – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_139_f5 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_141_zscaler – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_142_zscaler – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_145_tomcat – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_173_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_178_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_179_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_180_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_181_o365 – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_221_gcp – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_252_envoyproxy – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_253_envoyproxy – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_255_suricata – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_256_suricata – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_257_suricata – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_264_squid – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_271_okta – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest
Name: Build&Test / x-pack/filebeat-build / test_fileset_file_301_azure – x-pack.filebeat.tests.system.test_xpack_modules.XPackTest

kvch force-pushed the fix-filebeat-sourcestore-in-harvestergroup branch 2 times, most recently from 5223c8d to 43a9ccd Compare

March 31, 2021 11:08

kvch mentioned this pull request

Add test to check if Prospector sets metadata of files on updates in filestream #24868

Merged

1 task

kvch changed the title ~~Use sourceStore to HarvesterGroup in filestream~~ Use identifier in HarvesterGroup in filestream

urso approved these changes

View reviewed changes


          Use identifier in HarvesterGroup in filestream

356f306

kvch force-pushed the fix-filebeat-sourcestore-in-harvestergroup branch from 43a9ccd to 356f306 Compare

March 31, 2021 11:36

kvch added the needs_backport label

Contributor Author

kvch commented Mar 31, 2021 •

edited

Loading

Failing tests are unrelated, PR to fix it: #24871

kvch merged commit 23bfcd9 into elastic:master

kvch mentioned this pull request

Cherry-pick #24864 to 7.x: Use identifier in HarvesterGroup in filestream #24873

Merged

3 tasks

kvch added v7.13.0 and removed needs_backport labels

kvch added a commit to kvch/beats that referenced this pull request


          Use identifier in HarvesterGroup in filestream (elastic#24864)

076f67e

(cherry picked from commit 23bfcd9)

kvch added a commit that referenced this pull request


          Use identifier in HarvesterGroup in filestream (#24864) (#24873)

c7aa3bd

(cherry picked from commit 23bfcd9)

This was referenced Apr 1, 2021

Cherry-pick #24868 to 7.x: Add test to check if Prospector sets metadata of files on updates in filestream #24881

Closed

Cherry-pick #24868 to 7.x: Add test to check if Prospector sets metadata of files on updates in filestream #24883

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Team:Elastic-Agent v7.13.0